Linear regression. We have that the estimated mean in linear regression is. ˆµ Y X=x = ˆβ 0 + ˆβ 1 x. The standard error of ˆµ Y X=x is.

Transcription

1 Linear regression We have that the estimated mean in linear regression is The standard error of ˆµ Y X=x is where x = 1 n s.e.(ˆµ Y X=x ) = σ ˆµ Y X=x = ˆβ 0 + ˆβ 1 x. 1 n + (x x)2 i (x i x) 2 i x i. The estimated standard error of ˆµ Y X=x is 1 est.s.e.(ˆµ Y X=x ) = s Y X n + (x x)2 i (x i x) 2 where s Y X is the estimated standard deviation ( MSE) from the regression.. p.1/13

2 Confidence intervals The quantity ˆµ Y X=x (β 0 + β 1 x) est.s.e.(ˆµ Y X=x ) follows a t-distribution with n 2 degrees of freedom under the null hypothesis H 0 : µ Y X=x = β 0 + β 1 x. The (1 α)-confidence interval for µ Y X=x is thus ˆµ Y X=x ± t n 2,1 α/2 est.s.e.(ˆµ Y X=x ).. p.2/13

3 Prediction intervals The estimated mean, ˆµ Y X=x, can also be seen as a prediction, ŷ, of Y given X = x. For a new (independent) observation of Y given X = x the standard error of the residual of the prediction is s.e(y ŷ) = s.e(y ˆµ Y X=x ) = σ n + (x x)2 i (x i x) 2. The estimated standard error is computed by substituting σ with s Y X. The (1 α)-prediction interval is ŷ ± t n 2,1 α/2 est.s.e.(y ŷ).. p.3/13

4 Inverse prediction Prediction asks for the likely values of Y given X. Invers prediction asks for the likely values of X given Y. If we want to compute the prediction ˆx of X given Y = y we solve y = ˆβ 0 + ˆβ 1ˆx, that is ˆx = y ˆβ 0 ˆβ 1. Inverse prediction intervals are constructed by solving the equations y = ˆβ 0 + ˆβ 1 x t n 2,1 α/2 s Y X y = ˆβ 0 + ˆβ 1 x + t n 2,1 α/2 s Y X n n + (x x)2 i (x i x) 2 (1) + (x x)2 i (x i x) 2 (2) in terms of x. This gives two quadratic equations and results in upper and lower bounds on x in terms of y.. p.4/13

5 Inverse prediction The result is where x + ˆβ 1 (y ȳ) K ± t n 2,1 α/2 K (y ȳ) s 2 Y X i (x i x) 2 + K K = ˆβ 2 1 t2 n 2,1 α/2 s2 Y X i (x i x) 2. ( ) n. p.5/13

6 Regression and ANOVA In one-way ANOVA we study the influence of a single categorical variable (one factor) on the mean of our observations. In regression we study the influence of a single continues variable (the independet variable) on the mean of our observations. In one-way ANOVA we need one mean value parameter per group. In regression we need only a total of two parameters one giving the slope and one giving the intercept. The null hypothesis of equal group means in one-way ANOVA corresponds to the null hypothesis that the slope equals 0 in regression. The equal group mean hypothesis can be tested using the F -test (p. 327). The slope equal to 0 hypothesis can be tested with the t-test or equivalentely the F -test (p. 361,363,364).. p.6/13

7 Multiple regression The multiple regression model (section 13.4) with p 1 independent variables corresponds to a multifactor ANOVA with p 1 factors. The starting model (section ) is the additive model where we only take main effects of the independent variables into account. The F -test (p. 369) tests the null hypothesis that all p 1 slopes equal 0. The book does not discuss how to capture interaction in a regression setup (difficult).. p.7/13

8 Combining regression and ANOVA There is no problem in combining categorical factors and continues variables in a single analysis. No interaction between factors and continues variables: Intercept is determined by the combination of factors, but there is a single slope for each continues variable. Interaction between factors and continues variables: Slope as well as intercept for each continues variable depends upon the combination of factors. F -tests for no interaction between a factor and a continues variable, say, means that we test if the slope is the same for all groups corresponding to the factor.. p.8/13

9 One factor + regression If we have one factor with groups numbered 1,...,r and one continues variable we can specify the model y ij = µ + α i + β i x ij + ɛ ij, i = 1,...,r, j = 1,...,n i, x ij the value of the independent variable for the ij th observation. This model corresponds to the two-way ANOVA with interaction (we allow the slopes to depend upon the group.) The model y ij = µ + α i + βx ij + ɛ ij, corresponds to two-way ANOVA without interaction only main effects (the slope is the same for all groups). The model y ij = µ + βx ij + ɛ ij, is the regression model where there is no main effect of the group factor. p.9/13

10 ANOVA table È Source DF Sum of Squares Mean Square F -ratio Group r 1 SSG MSG = SSG/(r 1) MSG/MSE Regression 1 SSR MSR = SSR/1 MSR/MSE Interaction r 1 SSRG MSRG = SSRG/(r 1) MSRG/MSE Error (Residual) n 2r SSE= i,j (y ij ŷ ij ) 2 MSE = SSE/(n 2r) Total n 1 SST= È i,j (y ij ȳ.. ) 2 Total number of observations: n i 2 n = i=1 r n i ŷ ij = ˆµ + ˆα i + ˆβ i x ij.. p.10/13

11 Testing for equal slopes With r groups we want to investigate if the slopes are equal: H 1,0 : β 1 =... = β r against the alternative that (some) can differ. A test of H 1,0 is in the ANOVA table a test for the interaction of the group factor and the regression variable. The test statistic is MSRG MSE, which is compared to the F -distribution with (r 1, n 2r) degrees of freedom.. p.11/13

12 Testing for equal intersection With r groups we can also investigate if the intersections are equal: H 2,0 : α 1 =... = α r against the alternative that (some) can differ. A test of H 2,0 is in the ANOVA table a test for the main effect of the group factor. In contrast to general ANOVA it does make technical sense to test main effects (H 2,0 ) before interactions (H 1,0 ). (But if H 1,0 is true, the regression lines are all parallel and H 2,0 says that they are equal, if H 1,0 is not true, H 2,0 just says that all lines intersect in an arbitrary point). The test statistic is MSG MSE, which is compared to the F -distribution with (r 1, n 2r) degrees of freedom.. p.12/13

13 Testing linearity If we for all x-values have 2 or more observations it is possible to test linearity against a general one-way ANOVA model. If we have r groups of x-values, x 1,...,x r with n 1,...,n r all 2 y-observations in the groups, the total number of observations is n = r n i. i=1 È Source DF Sum of Squares Mean Square F -ratio Regression 1 SSR MSR = SSR/1 MSR/MSE Group r 2 SSG= i,j (ȳ i ŷ i ) 2 MSG = SSG/(r 2) MSG/MSE Error (Residual) n r SSE= i,j (y ij ȳ i ) 2 MSE = SSE/(n r) Total n 1 SST= È i,j (y ij ȳ.. ) 2 ŷ i = ˆα + ˆβx i. p.13/13